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Abstract. Minimization of boundary curvature is a classic regularization tech- 
nique for image segmentation in the presence of noisy image data. Techniques 
for minimizing curvature have historically been derived from descent methods 
which could be trapped in a local minimum and therefore required a good initial- 
ization. Recently, combinatorial optimization techniques have been applied to the 
optimization of curvature which provide a solution that achieves nearly a global 
optimum. However, when applied to image segmentation these methods required 
a meaningful data term. Unfortunately, for many images, particularly medical 
images, it is difficult to find a meaningful data term. Therefore, we propose to 
remove the data term completely and instead weight the curvature locally, while 
still achieving a global optimum. 



1 Introduction 

A classic prior model for the boundary of objects in an image is that the boundary have 
a small curvature. This model was proposed and theoretically justified by Mumford 1 1 1 
and also appeared in the first active contour work by Kass, Witkin and Terzopoulos |2| 
who proposed an optimization of the boundary curvature. Subsequently, the optimiza- 
tion of boundary curvature became a common feature of variational methods for active 
contours and level sets [3 4 5]. However, all of these methods use descent-based opti- 
mization, causing the solution to get stuck in a local minimum and depend strongly on 
having a good initialization. Following the description by Bruckstein et al. [6| of the 
curvature of a polygon, the curvature of an object boundary was formulated on a graph 
by Schoenneman et al. [7 1 (and previously in a different manner by the same authors 
[8 1). Specifically, on the graph dual to the pixel lattice, the boundary of an object may 
be described by a polygon comprised of graph edges, and therefore the boundary having 
minimum curvature could be found by optimizing over all polygons which had a cur- 
vature value as defined by Bruckstein et al. . However, this optimization required a long 
computation time (minutes to hours) and often did not find a solution achieving a global 
optimum. Therefore, this work was followed by El-Zehiry and Grady [9| who used a 
primal formulation of the curvature energy that parameterized the boundary polygon in 
terms of the normal vectors and computed a solution in seconds which often achieved 
a global optimum. For simplicity, in this work we consider only the segmentation of an 
object from a background, i.e., a two-class image segmentation problem. 
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All of these previous approaches to utilizing curvature minimization employed cur- 
vature as a boundary regularization for a data term which modeled the object and back- 
ground intensity/color/texture/etc. Since this data term was imperfect (due to noise), 
the curvature regularization was used to regularize the solution. However, in practice, 
this approach can suffer from two problems: 1) It may be hard to model the object and 
background data a priori or, worse, the distributions can significantly overlap (or even 
be equal), 2) Sometimes the true object boundary has a high curvature, but a minimum 
curvature regularization can cause the boundary to become erroneously smooth (e.g., 
by cutting of corners). In particular, the data modeling problem often occurs in medical 
imaging. For example, a tumor may be made of the same tissue as the organ it is at- 
tached to, such that its intensity distribution appears identical to the organ's intensity in 
the acquired image (meaning that only spatial information can be used to distinguish tu- 
mor from healthy tissue). Another example is in the segmentation of heart chambers in 
which each chamber is filled with blood (i.e., has the same appearance) and, worse, an 
open valve between the chambers means that only the spatial information is capable of 
distinguishing one chamber from another. Consequently, in this paper we highlight the 
ability of our method to operate in these difficult circumstances encountered in medical 
imaging, with the understanding that the technique could be applied to any arbitrary 
images encountered in computer vision. 

One method which has been used in past to address the unreliability of a data term 
is to simply employ contrast-dependent weights on the graph edges, which requires 
the optimization of only a single term. However, to avoid a trivial solution, previous 
methods have employed seeds; a foreground seed is a small subset of pixels that have 
been labeled as belonging to object and a background seed is a small subset of pix- 
els that have been labeled as belonging to background. These seeds may be obtained 
interactively from a user who is specifying a particular object (e.g., 01011111211 ) or au- 
tomatically from a system trained to look for a particular object (e.g., B13I141 '). For 
example, this type of approach which uses seeds and contrast-sensitive edge weighting 
has been employed with such popular algorithms as graph cuts ifTOl , random walker 
[ 1 5 1 , geodesic segmentation [ 1 6 1 and power watersheds ifTTl . We propose to apply this 
same approach in the curvature method of [9], which has the effect of allowing object 
boundary curvature to be high when supported by image contrast, but penalizing bound- 
ary curvature for weak or noisy object boundaries. In this way, our formulation solves 
both of the problems outlined above that can be encountered when a curvature term is 
used as regularization for a data model, since the formulation does not require a reliable 
data prior and also permits high-curvature boundaries when supported by data. 

The use of a weighted curvature objective function is designed to overcome prob- 
lems with unreliable data terms and permit a high-curvature object boundary when 
supported by image contrast. Unfortunately, the authors of [9| perform the optimiza- 
tion of object curvature on a graph using the Quadratic Pseudo-Boolean Optimization 
with Probing (QPBOP) method, which can be expected to produce all unlabeled pixels 
when there is no unary (data) term B18I19L Therefore, QPBOP will not allow us to find 
a global optimum of the weighted curvature functional when no data term is present. 
However, we show that by introducing an attraction force , we can decrease the number 
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of negative weighted edges in the formulation presented in [9 1 which allows QPBOP to 
find the optimal solution. 

After describing our image segmentation technique, we demonstrate that it can be 
applied to control images that were designed to have zero difference between the in- 
tensity distribution of the object and background, as well as having large sections of 
missing boundary information. Although other techniques are known for robustness to 
weak boundaries, such as graph cuts and random walker, we show that they are unable 
to address these difficult cases, while our algorithm can. Older active contour methods 
could also be applied to solve some of these image segmentation problems, but they 
rely on a good initialization and a setting of parameters to tradeoff between multiple 
terms which often needed to be set individually for each image. Following these control 
images, our algorithm is applied to a series of medical image segmentation problems 
that exhibit the same challenges which were observed in the control set. 

2 Methods 

We begin this section with a review of the 2D curvature optimization framework pre- 
sented in [9] before proceeding to our contrast weighted curvature formulation, opti- 
mization and addition of an attraction force. 

2.1 Curvature Energy 

The continuous formulation of Mumford's Elastica model is defined for curve C as 



Jc 

where k denotes the scalar curvature and ds represents the arc length element. When 
a = (the arc length is ignored), the model reduces to the integral of the boundary 



The use of combinatorial optimization by 1 9 ] to minimize the elastica model prompted 
the discrete formulation of the curvature on a graph. A graph Q = {V, £ } consists of a 
set of vertices v € V and a set of edges e e £ C V x V. An edge incident to vertices 
Vi and Vj is denoted ey. In our formulation, each pixel is identified with a node, Uj. A 
weighted graph is a graph in which every edge is assigned a weight Hy. An edge cut 
is any collection of edges that separates the graph into two sets, S C V and S, which 
may be represented by a binary indicator vector x, 




(1) 



squared curvature E(C) = J c 



2 ds. 




(2) 



The cost of the cut represented by any x is given by 




(3) 



Bruckstein et al. [6 1 expressed the curvature of a 2D polygon in terms of the angular 
change between consecutive polygonal segments. 
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Instead of a polygon, it was observed in [7| that the polygon could be viewed as 
existing on a dual graph. However, in [9| the same idea was formulated on a primal 
graph in order to permit an easier optimization and more generalizable formulation. In 
this formulation, if two edges and e^, incident to a node Uj, are cut then the cut is 
penalized with value = =§ — =f — , where a is the angle between the edges. 

™™(l|ei 3 IU|e;«ll) 

This cut penalty was then exactly decomposed into three edge weights 

E(xi,Xj,x k ) = w ij \x l - Xj\ + w ik \xi ~ x k \ - w jk \xj - Xfc|, (4) 

where uiy = w^k — Wjk = \wijk- Therefore, the minimum cut with respect to these 
edge weights is a cut that minimizes Bruckstein's discrete curvature formulation on 
the dual graph. Despite the negative weights, it was shown in |9| that QBPOP was 
able to find a minimum cut in most circumstances. Notice that although the curvature 
clique was designed to penalize the cut of both edges ey and e^., the decomposition 
to pairwise interactions add an edge e,-fc with negative weight. We denote the the set of 
effective edges with nonzero weights as £* D £ . 

2.2 Weighted Curvature 

To perform curvature based segmentation in [7 9 1, data term had to be added to the cur- 
vature model to obtain the boundary of the object of interest. To solve the segmentation 
problem without dependence on a data term, we propose to weight the curvature term 
locally based on the image intensity profile. The primal pairwise curvature formulation 
in J9 j makes it feasible to weight the curvature based on the intensity differences be- 
tween the terminal points of a particular edge. 




curvature Contrast weighted 

curvature 



Fig. 1. Weighted curvature clique versus unweighted curvature clique. 

According to the curvature formulation in (9), edges ey and are cut when the 
pixel i is a foreground pixel and j and k are background pixels or vice versa. Therefore, 
the curvature clique formed by these edges should be weighted by the intensity differ- 
ences between pixels i and j and the intensity difference between i and k. 
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This can be formulated as follows: Given a 2D image with image values associated with 
each pixel (node), g : V — > R. The weighted curvature uiCijk is given by 

wci jk = w ijk w 1 ^ w' ik , (5) 

where 

^ = exp(-j3(jg(i) - g(j)) 3 ), (6) 

w' ik = exp{-p{g{i)-g{k)) 2 ). (7) 

The parameter (3 > controls the contract strength. The cut penalty is calculated using 
the same decomposition in|4]with weights Wij=Wik=Wjk= wc ^ h depicted in Figure 1. 

The contrast weighted curvature regularization eliminates the dependance on the 
data model and provides better segmentation of high curvature features in the image 
(such as sharp corners) when supported by high contrast. 



2.3 Optimization 

In the previous sections, the original Bruckstein formulation of polygonal curvature was 
transformed into the problem of finding a minimum cut on a graph in which some of 
the edge weights were negative. Unfortunately, the negative edge weights introduced by 
the third term of Q causes the minimum cut problem to be nonsubmodular |20|, i.e., 
straightforward max-flow/min-cut algorithms will not yield a minimum cut. However, 
it was shown in |9 1 that the Quadratic Pseudo Boolean Optimization (QPBO) 1 18 1 and 
Quadratic Pseudo Boolean Optimization with Probing (QPBOP)| 19 1 offered a solution 
to the optimization problem that frequently offered a complete, optimal solution. 

A practical problem that we encountered due to the elimination of the data model 
(unary term or terminal links in the graph) is that the optimization using QPBO/ QPBOP 
(used in [9|) fails to provide complete labeling. Hence, in the next section present the 
attraction energy that enables the QPBO/ QPBOP to provide complete labeling. 



2.4 Attraction Energy 

The second contribution of this paper is the introduction of the attraction force to the 
segmentation problem. The attraction force is inspired by the intermolecular forces that 
maintain the structure of the molecule and prevent it from decomposing into its salient 
atoms. This scenario is very analogous to the segmentation problem where the object 
to be segmented is similar to a molecule that we would like to maintain its atoms (the 
pixels that constitute the object of interest) tightly connected to each other. Coulomb's 
Law states that the force of attraction between two objects is equal to the product of 
their charges qi and q 2 is given by 

F=^ qi q 2 , (8) 

where k is the Coulomb's constant and r is the distance between q\ and q 2 . Hence in our 
discrete formulation, we need to maximize the attraction force between two variables 
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X\ and X2 given by 

^attraction = ^(^1^2 + (1 ~ ~ ^j^J, (9) 

where w\2 is the weighted curvature weight of the edge ei2 described in the previous 
section. Notice that maximizing this attraction force is equivalent to minimizing its 
negative. 

A key value of the attraction force is that it allows for an optimization of the cur- 
vature energy in the absence of the unary term. Specifically, the construction in [20 1 
represents the negative attraction energy by adding edges ei2with weight W\2 , esi an 
edge C2T with weights The addition of positive weights changes the sign of some 
of the negative weights introduced by the curvature term. This justifies our choice for 
the attraction weight as which makes the the weight of ej.2 nonnegative. These sign 
changes affect the optimization problem by strongly decreasing the number of unla- 
beled variables in the output of the QPBOP. 

2.5 Summary 

The segmentation problem is modeled as the solution, x, which minimizes the energy 

E(x) = E c 

urvaturc 

(x) - A ^attraction (x) , (10) 

for strictly positive weighting parameter A that controls the relative importance of the 
attraction energy with respect to the curvature energy. 
The curvature term is written as a summation by 

-^curvature (^) — ^ ^ ^ij\Xi Xj | , (11) 

and the attraction force is given by 

^attraction (%) = ^ -^( X i X j + i 1 ~ ^i 1 ~ X j))' ( 12 ) 

Foreground and background seeds are used to initialize the segmentation, a fore- 
ground seed Vi is set to Xi = 1 while a background seed would be set to Xj = 0. 

3 Experimental Results 

This section presents a sample of our segmentation results using weighted curvature 
and compares our segmentation to the corresponding results obtained by graph cuts 
and random walker. In all of our results, we have used the 8-point connected lattice. 

1 Note that the addition of the second term here is very natural and still analogeous to ([8} in 
the sense that we have two molecules, one for the foreground and one for the background 
and hence we have to add two terms for the attraction force; one penalizing both pixels to be 
labeled 1 and another to penalize both pixels being labeled 
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We begin by demonstrating the usefulness of weighted curvature on synthetic images. 
These synthetic images feature two important challenges that can only be resolved by 
weighted curvature regularization: 1) Absence of data differences at some parts of the 
boundary, i.e., object and background have the same intensity profile. 2) Segmentation 
of high curvature features such as cusps and sharp corners. 

Figure 2 shows three images that the human visual perception can seamlessly define 
where the boundaries of objects of interests are. However, some of the most advanced 
segmentation tools can not. For example, in the first image of Figure 2, it is obvious 
that the blue seed should provide an object that ends where the white lines end. Graph 
cuts favor the cut with the minimal number of edges since there no contrast between the 
neighboring pixels which yields a trivial solution isolating the blue seed as a foreground 
and the rest of the image as a background, graph cuts also gave trivial solutions in the 
rest of the images. Random walker works, intuitively, by calculating the probability that 
a random walk starting at a particular pixel will first reach one of the seeds. Hence, it 
suffers from a proximity problem that results in a premature stopping and segments 
a bar with a smaller length than the correct one. Our approach provides the correct 
segmentation. Although the seed is very far from the end of the bar, our algorithm could 
extend the segmentation until the end of the bar. This is simply because a a straight line 
has a minimal curvature so the algorithm extends the bar until an intensity difference 
occurs. 

The second image features a disconnected boundary. This challenge is common 
in real images when acquisition artifacts such as noise and occlusion disconnect the 
boundary. In medical images, anatomical abnormalities such as stenosis in vessels may 
cause the boundary to appear disconnected. The random walker leaks through these 
gaps and fails to separate the object from the background correctly. Weighted curva- 
ture, however, succeeds to bridge the gaps in the boundary due to the unique ability 
of curvature to preserve object continuity as connected boundaries have less curvature 
than disconnected ones. Meanwhile, the sharp corners were not smeared by curvature 
minimization because they were supported by contrast and the curvature is weighted 
by this contrast information in our formulation. The third image consists circle with a 
bump (of the same intensity) attached to it. The large circle is the object of interest that 
should be separated from the smaller bump and the black background. This scenario 
is very common in medical images. For example, a tumor may be made of the same 
tissue as the organ it is attached to, with no intensity differences between the tissue 
and the tumor (see Figure 6) or two proximal distinct structures may have the same 
intensity profile (such as the caudate and the putamen in brain MRI, for example). In 
figure 2, random walker stops prematurely yielding a smaller circle than the correct one 
because a random walk from an erroneously-background-labeled pixel would have a 
higher probability reaching the background seeds than the foreground one. However, 
weighted curvature succeeds to obtain the correct boundary; the contrast based weights 
forces the boundary to stop when it hits a black pixel isolating the large circle from the 
background. And the curvature favors a circle with larger radius and hence it extends 
the foreground seeds until it reach the boundary between the large circle and the smaller 
bump. The foreground can not be extended any further, otherwise, it would form a cusp 
at the blue seed in the bump yielding a high curvature. Premature segmentation is pro- 
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Fig. 2. Segmentation of some challenging images. Graph cuts provide a trivial solution 
by isolating the foreground or the background seeds. Random walker suffers from prox- 
imity problem that yields a smaller object than the desired in images 1 and 3. It also 
suffers from a leakage producing a segmentation of the second image due to the gaps in 
the boundary. Weighted curvature extends the bar in the first image at no cost and stops 
when contrast change occurs. It bridges the gaps in the second image because curvature 
preserves connectivity. In the third image, the curvature regularization separates two 
structures of the same intensity. 

hibited by the fact that a circle with a larger radius has a smaller curvature thana circle 
with a smaller radius. Figure 3 shows similar examples with larger gaps in the bound- 
aries. Notice that although the gaps in the dotted curve (in the first image) are very 
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Fig. 3. Segmentation of boundaries with very large gaps and zero contrast between fore- 
ground/background pixels. The gaps caused leakage in the random walker that failed 
to provide the desired segmentation. Curve continuity is promoted by curvature regu- 
larization that enables our approach to bridge the gaps in the boundaries of the three 
images. Our approach also maintains the cusps in the first two images because they are 
warranted by the contrast weighted edges. 

small, these gaps caused the random walker to leak. Also, random walker could not 
connect the large gap, between two end points of the curve, by a straight line. Weighted 
curvature, however, prefers the straight line because of its minimal curvature which 
wields the desired segmentation. The second and third images in Figure 3 also feature 
very large gaps in the boundaries with no foreground/background contrast, these gaps 
challenged the random walker resulting in erroneous segmentation. Weighted curvature 
succeeds to bridge all the gaps in the boundaries. Notice that the cusps in the first im- 
age and the sharp corners in the second image were segmented correctly because the 
curvature weights are contrast dependent. 

The previous challenges occur frequently in medical images. Here we demonstrate 
the performance of our model on several medical imaging applications and different 
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Fig. 4. First column: Segmentation of the left Atrium. Second Column: Segmentation of 
the left ventricle. Graph cuts yielded a trivial segmentation around the seeds. The lack 
of contrast between the left atrium and the background, in addition to the proximity of 
several parts of the images to the foreground seeds forced the random walker to leak and 
produce an overcompensation of the region of interest in both cases. Weighted curvature 
prevented leakage because leakage introduces higher curvature cost of the boundary. 
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Fig. 5. Segmentation of circumscribed masses in mammograms. Graph cuts resulted 
in a trivial solution by cutting the edges around the foreground seeds. Random walker 
resulted in false negatives in the segmentation of the first mass and false positives in the 
segmentation of the second one because the probability of a random walk reaching a 
foreground or background seed is affected by the location of the seeds yielding a false 
segmentation in both cases. Weighted curvature detected the correct boundaries in both 
cases. 
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Fig. 6. First column: Segmentation of a lung tumor in a chest CT scan. Graph cuts pro- 
duced a trivial segmentation around the foreground seed. Random walker did not ex- 
tract the whole tumor due to the proximity of some tumor pixels to background seeds. 
Weighted curvature correctly classified the pixels that random walker missed due ti 
their high contrast with the lung tissue. Second column: segmentation of the liver in an 
abdominal MRJ slice. Random walker yielded false positives and false negatives and 
disconnected the liver into two pieces while weighted curvature preserved the connec- 
tivity of the liver and captured the correct boundary. 
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modalities. Figure 4 shows the segmentation of the left atrium in ultrasound. The atrium 
has the same intensity profile as the rest of the heart chambers and the background 
with no clear boundaries. Graph cuts yielded a trivial solution isolating the foreground 
seeds as the object and the rest of the image as a background. The absence of image 
contrast between the atrium and the background resulted in an over-segmentation by 
random walker. The segmentation using our proposed approach succeeded to extract 
the left atrium correctly. The second columns of Figure4 exhibits another example for 
such challenges where the left ventricle is the object to be segmented. Mass segmen- 
tation in mammograms is a challenging problem that can not easily be modeled by 
data terms because of the large variability among the different types of breast tissues 
(e.g fatty, glandular and dense tissues) and because of the variability in the appear- 
ance and shape of the masses . Figure 5 demonstrates the segmentation of two circum- 
scribed masses in mammograms. For the first mass, the random walker result suffers 
from false negatives (under-segmentation) while the second mass the random walker 
provided false positive pixels (over-segmentation). In both cases, the weighted curva- 
ture yielded a correct segmentation. Figure 6 depicts the segmentation of a lung tumor 
in a chest CT scan. There is no contrast between the tumor and abdominal muscles. 
Random walker stopped prematurely (due to the proximity issue) and did not extract 
the whole tumor while our segmentation did. The weighted curvature expands the fore- 
ground minimize the curvature of its boundary and it stops when it reaches the boundary 
between the tumor and the muscles that produces the least curvature. The second col- 
umn of figure 6 shows a similar case where there is no contrast between the liver and 
the abdominal muscles in an abdominal MR scan. Random walker yielded some false 
positives and and false negatives. For example, the upper boundary of the liver was dis- 
connected by random walker. It is intuitively clear that random walks starting at these 
erroneously -background-labeled pixels (the gap between the two pink parts captured by 
random walker) have higher probability to reach background seeds than foreground and 
hence falsely labeled as background. However, curvature regularization favors a smooth 
boundary over a disconnected one because disconnecting the upper boundary into two 
pieces would form corners with high curvature. All the previous results were obtained 
using the same parameters for attraction force and contrast weighting. Complete global 
solution was obtained in each case. The addition of the attraction force enabled the 
QPBO to provide complete labeling without any need to perform probing which makes 
our algorithm even faster than (5). In the few cases that QPBO did not label all the 
pixels, the probing was able to provide these pixels yielding a global optimal solution. 

4 Conclusion 

We proposed an image segmentation method which isolates an object boundary by us- 
ing weighted curvature. Weighted curvature allows us to segment an object in challeng- 
ing situations in which a data term is unreliable or missing, as is common in medical 
image segmentation tasks. Additionally, our formulation of weighted curvature permits 
the segmented object boundary to exhibit a high curvature if it is warranted by the im- 
age data. The inclusion of an attraction force allows us to find a global optimum of the 
model using QPBOP in a few seconds. 
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Our experiments demonstrate that our model can be used to complete object bound- 
aries in controlled experiments where the object/background shared the same inten- 
sity distribution and significant parts of the boundary were missing. In contrast, other 
leading algorithms which are known for this type of robust behavior were unable to 
achieve segmentations of these challenging control images. Similarly, our algorithm 
was demonstrated to provide quality segmentations on image segmentation tasks ap- 
pearing in medical images which exhibit similar challenges to the control images. Once 
again, we demonstrate that graph cuts and random walker are unable to perform with 
such little information (in terms of image data and seed numbers/placements). In all of 
these experiments, both control and real, the same parameters were used for the contrast 
weighting and attraction force term. 

Future work will be to investigate the use of other optimization methods that might 
allow us to remove the attraction force (and consequent parameter), the use of a data 
term in conjunction with weighted curvature and to take advantages of recent advances 
in GPU-based graph cuts (and therefore QBPOP) to further accelerate the already fast 
speed of our method. 
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